A REMARK ON IRREGULARITY OF THE <9-NEUMANN 
PROBLEM ON NON-SMOOTH DOMAINS 
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Abstract. It is an observation due to J.J. Kohn that for a smooth bounded pseu- 
doconvex domain Q in C n there exists s > such that the (9-Neumann operator on 
maps W* ^(O) (the space of (0, l)-forms with coefficient functions in L 2 -Sobolev 
space of order s) into itself continuously. We show that this conclusion does not 
hold without the smoothness assumption by constructing a bounded pseudoconvex 
domain fl in C 2 , smooth except at one point, whose 9-Neumann operator is not 
bounded on W* for any s > 0. 

Let IV s (fl) and W^ pq - ) (Q) denote the L 2 -Sobolev space on Q of order s and the 
space of (p, g)-forms with coefficient functions in W S (Q), respectively. Also ||.|| s ,n 
denotes the norms on W* pq ^(Q). Let N q denote the inverse of the complex Laplacian, 

dd + d d, on square integrable (0, g)-forms. It is an observation of Kohn, as the 
following proposition says, that on a smooth bounded pseudoconvex domain the 
9-Neumann problem is regular in the Sobolev scale for sufficiently small levels. 

Proposition 1 (Kohn). Let Q be a smooth bounded pseudoconvex domain in C n . 
There exist positive e and C (depending on Q) such that 

||iVi|| £) n < C||?/|| £) n, \\dN q u\\ £t n < C||u|| £) n, \\d*N q u\\ £ , n < C||u|| e) n 
for u e W? 0g Jfl.) and 1 < q < n. 

We show that if one drops the smoothness assumption then the 9-Neumann oper- 
ator, N±, may not map any positive Sobolev space into itself continuously. 

Theorem 1. There exists a bounded pseudoconvex domain Q in C 2 , smooth except 
one point, such that the d-Neumann operator on Q is not bounded on W* Q ^ (Q) for 
any s > 0. 

Proof. We will build the domain by attaching infinitely many worm domains (con- 
structed by Diederich and Fornasss in |DF77j ) with progressively larger winding. Let 
Qj be a worm domain, a smooth bounded pseudoconvex domain, in C 2 that winds 
2%j such that 

Qj C {(z,w) e C 2 : \z\ < 2~ j ,4T ] < \w\ < 4" J 2} 
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for j = 1,2, . . .. Let be a straight line that connects an extreme point on the 
cap of Qj to a closest point on the cap of Qj+i. Then using the barbell lemma (see 
[FS77t IHW68] ) we get a bounded pseudoconvex domain Q that is smooth except 
one point (0, 0) G bQ. Notice that Q is the union of Qj C Q for j = 1,2,... and 
all connecting bands. In the rest of the proof we will show that if the 5-Neumann 
operator on Q is continuous on W? 01 s(Q) then the <9-Neumann operator on Qj is 
continuous on Wf 01 JQj) for j = 1,2,.... However this is a contradiction with 
a theorem of B arrett ( [B ar 9 2 j ) . Let us define D- 7 = dd* + d*d on L3 01 ^(Qj), and 
□ = dd* + d*d on L? 01 JQ). Let us fix j and choose a defining function p for Qj 

such that || Vp|| = 1 on bQj. Let v = Re \^2^=i ^r^~j an d J denote the complex 

structure of C 2 . Now we will construct a smooth cut off function that fixes the 
domain of □ and d under multiplication. We can choose open sets U±, U 2 , and U3 
and x £ C™(U 2 ) such that 

i) U, CC U 2 CC U 3 , 

ii) U\,U 2 , and U3 contain all boundary points of Qj that meet the (strongly 
pseudoconvex) band created using 7^ and 7 3 -_i, and they do not contain any 
weakly pseudoconvex boundary point of Qj, 

iii) < % < 1, X = 1 on U x , 

iv) there exists an open set U such that bQj U U 2 CC [/ and the following two 
ordinary differential equations can be solved in U 

(1) KV0 = o, = x , 

(2) v$) = -J(v)(x), ?|mi s -=0. 

Notice that if; = 1 and = on f/i, and -0 = = in a neighborhood of the set of 
weakly pseudoconvex boundary points of Qj. We choose a neighborhood V CC U 
of M7j and % £ Co°(V) such that x = 1 in a neighborhood V of MX,-. Let us define 

(f) = x<fi, if) = x^i an d £, = ifi + i<p- We like to make some observation about £ that 
will be useful later: 

i) f = 1 on V DZTl, 

ii) (i/ + «J(z/))(f) = on 6%, 

iii) £ = in a neighborhood of the weakly pseudoconvex boundary points of Qj. 

Claim: If f G Dom(oi) then £f G Dom(D J ') and (1 - £)/ G Dom(D). 

Proof of Claim: First we will show that £/ G .Dora(Q ? ') then we will talk about how 
one can show that (1 — £)/ G Dom(D). 

One can easily show that £/ G Dom{d )HDom(d) (on Qj). On the other hand, by 
Kohn-Morrey-Hormander formula [CSOlj since the L 2 -norms of any "bar" derivatives 
of any terms of / on Qj is dominated by + 1|<9 we have d (£/) G Dom(d). 

So we need to show that d(£f) = d£ A / + £df G Dom(d*). Since £3/ G Dom(d ) 
we only need to show that <9£ A / G Dom(d ). We will use the special boundary 
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frames. Let 



dp d dp d 



dp d dp d 



dzi dz 2 dz 2 dz\ ' dz\ dz 1 dz 2 dz 2 

Also let w T and w u be the dual (1, 0)-forms. We note that L v = v — iJ{y) and so 
L V {E) = on bQj. We can write / = f T W T + f v w u . Therefore, d£ A / = (L T (£)f v — 
L u (£)f T )w T AW U . Using the fact that f v £ Wq^Qj) (it is easy to see this for / £ 
C 1 ^). For / £ Dom{8*) n Dom(d) one can us e the fact that A : W^Qj) -> 
W _1 (r2j) is an isomorphism and the density lemma [CSOlt Lemma 4.3.2] to see this) 
and L T (£) is smooth we may reduce the problem of showing d£ A / £ Dom(d ) to 
show the following 

Lv(i)f T w T AW U E Dom(d ). 
Let be a sequence of smooth compactly supported functions converging to 

L v {£) in C 1 -norm and u be a (0, l)-form with smooth compactly supported coefficient 



functions in fL. Then 



(L u (£)f T w T A w v ,du) n . 



lim ( 

k— >oo 



where is the inner product on forms on Qj. If we integrate by parts and use 

lim^oo \\Li(<f) k f T )\\ nj = \\Li(L u (^)f T )\\ nj for I = r, v we can reduce the problem of 
showing d£ A / £ Dom(d*) to showing that \\£-(L v ^)f T )\\ n . and 11^(1,(0^)11^ 
are finite. One can show that 



d 



dZ T , 



lim 

k— >oo 



d 



dZr, 



lim 

k— >oo 



7p-(0fc/r) 



On the second equality we used integration by parts. On the other hand, we have 



lim 

k— >oo 



d 



dZr, 



Jr) 



L/4trn. 



d 



d 



< 



C(||5/|| n , + ||57||n,)<oo 

for m = 1,2 and a positive constant C that does not depend on /. In the last 
inequality we used the fact that L 2 -norms of / and the "bar" derivatives of f T on 
Qj are bounded by C7(||9/||n, + \\d fWuj)- We remark that it is essential that £ is 
complex valued and Q is smooth in a neighborhood of Qj. Therefore, we showed that 
£f £ Dom(ui). 

As for (1 — 0/ being in Dom(D). Since £ = 1 in a neighborhood of the boundary 
points of flj that meets the band created using jj and 7^_i we have (1 — £)/ = 
on f2 \ Also since Lj,(l — £) = —L u (^) similar calculations as before show that 
(1 — £)/ ^ -Dom(D). This completes the proof of the claim. 

We will use generalized constants in the sense that ||A|| S) ^ < ||-B|| S) n. means that 
there is a constant C = C(s, Qj) > that depends only on s and Qj but not on A or 
-B such that ||A|| Sj n- < C||-B|| S| q.. Assume that the 9- Neumann operator on Q maps 



4 



SONMEZ §AHUTOGLU 



W* 0l j(Q) into itself continuously for some s > 0. That is, ||iVi/i|| S) n < ||/i|| Sj n f° r 
h gV ( s 01) (Q). Then we have ||^|| S) n < ||n^|| s , n for g G Dom{U) and Ug e W^ 1} (Q). 
Let / e Dom{ni) and G Wfa A) (Qj). Then we have 

< ll£/IU + ll(i-£)/IU- 



Since £ = in a neighborhood of the weakly pseudoconvex boundary points of f2j 
we can use pseudolocal estimates on Qj (see |KN65j ) to get 



(3) Ufhfl ,^ll° i /ll^i,n, + H ni . 



Let us choose rj to be a smooth compactly supported function that is constant 1 
around the support of V£ and zero in a neighborhood of the weakly pseudoconvex 
points of Qj. Therefore, we have 

11(1 = 11(1 -OflUn < l|n(i-0/ll.,o 

< ||(A0/||. lO + ||Ve • V/H.,0 + 11(1 - A /|| S)Q . 



< 
< 



The first inequality comes from the assumption that the d- Neumann operator on Q is 
continuous on W? Q ^(Q)- The second inequality comes from the fact that □ operates 
as Laplacian componentwise on forms. In the last inequality we used the pseudolocal 
estimates as we did in (13]). Therefore we showed that 



Ln, <||e/l| Sl n, + ||(l-C)/l| Sl n,<||n J /|| s ,a, 

for / G Dom(D J ) and □■?/ G W^ 01 ^(Qj). One can check that this is equivalent to the 
condition that the d- Neumann operator on Qj is continuous on WL u(Qj)- D 

One can check that d N± maps W? 01 \(Q) into W S (Q) continuously if and only if 

PVlUfi ^ ll D /IUn for / G Domip) and □/ G W^iQ). Using this observation 
one can give a proof, similar to the proof of the theorem, for the following corollary. 

Corollary 1. There exists a bounded pseudoconvex domain Q in C 2 , smooth except 
one point, such that d N\ is not bounded from ^(Q) into W S (Q) for any s > 0. 

It is interesting that for a smooth bounded pseudoconvex domain Q in C 2 the 
operator dNi is bounded from W? 01 JQ) into Wf 02 JQ) for any s > 0. (One can use 
(4) in [BS90] to see this). 

Remark 1 . We would like to note the following additional property for the domain we 
constructed in the proof of Theorem [U There is no open set U that contains the non- 
smooth boundary point of Q such that U D Q has a Stein neighborhood basis. That 
is, non-smooth domains may not have a "local" Stein neighborhood basis. However, 
this is not the case for smooth domains (see for example [Ran86t Lemma 2.13]). 
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